<html>
<head><meta charset="utf-8"><title>reserved prefixes · edition 2021 · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/index.html">edition 2021</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html">reserved prefixes</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="232830802"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232830802" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232830802">(Apr 01 2021 at 21:57)</a>:</h4>
<p>(logically splitting off of the <code>k#keywords</code> topic)</p>
<p>I've filed RFC 3101 (<a href="https://github.com/rust-lang/rfcs/pull/3101">https://github.com/rust-lang/rfcs/pull/3101</a>) as the generalized syntactic reservation of the sort that RFC 3098 (<code>k#keywords</code>) is looking to leverage. If accepted this would no longer require RFC 3098 to be an edition change.</p>



<a name="232834600"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834600" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834600">(Apr 01 2021 at 22:35)</a>:</h4>
<p>I'm very much in favor of that RFC.</p>



<a name="232834605"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834605" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834605">(Apr 01 2021 at 22:35)</a>:</h4>
<p>One question:</p>



<a name="232834647"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834647" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834647">(Apr 01 2021 at 22:35)</a>:</h4>
<p>Is there any reasonable way we could <em>prevent</em> macros or similar from using such tokens if they don't actually exist in the language?</p>



<a name="232834712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834712">(Apr 01 2021 at 22:36)</a>:</h4>
<p>I'm concerned that someone will go "oooh, a reserved space, I can use that for my macro meta-language's tokens".</p>



<a name="232834762"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834762" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834762">(Apr 01 2021 at 22:36)</a>:</h4>
<p>And then write something like <code>k#mykeyword</code> and then match <code>k#mykeyword</code>, or write <code>m#mything</code> and match <code>m#mything</code>.</p>



<a name="232834788"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834788" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834788">(Apr 01 2021 at 22:37)</a>:</h4>
<p>I acknowledge that the ability to do that would <em>also</em> be the ability to implement prototypes of language features as proc macros.</p>



<a name="232834809"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834809" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834809">(Apr 01 2021 at 22:37)</a>:</h4>
<p>But on the other hand, the ability to do that is also the ability to accidentally or intentionally use something in a way that'll break in the future if the language wants to use the same syntax, which is the point of reserving it.</p>



<a name="232834881"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232834881" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232834881">(Apr 01 2021 at 22:38)</a>:</h4>
<p>C <em>claimed</em> to reserve <code>__</code>, but in practice, many many libraries and operating systems and similar used <code>__</code> in their own "internal use" tokens for much the same reason C reserved it.</p>



<a name="232835239"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835239" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835239">(Apr 01 2021 at 22:42)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/232834712">said</a>:</p>
<blockquote>
<p>I'm concerned that someone will go "oooh, a reserved space, I can use that for my macro meta-language's tokens".</p>
</blockquote>
<p>AIUI, this is partly what the existing notion of "literal suffixes" (linked from the RFC) are for. i.e., the lexer provides <code>"foo"bar</code> or <code>123qux</code> as a single token, even though that has no meaning in normal Rust code.</p>



<a name="232835319"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835319" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835319">(Apr 01 2021 at 22:43)</a>:</h4>
<p>Sure. But that doesn't <em>prevent</em> people from using the <code>ident#</code> or <code>ident"</code> spaces for themselves.</p>



<a name="232835420"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835420" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835420">(Apr 01 2021 at 22:44)</a>:</h4>
<p>Furthermore, with the aid of this RFC, we could literally carve out such a space. The prefix <code>my#</code> (or w/e) could be "reserved for private use" in the same way that unicode reserves the private use area: "we give you permission to use this, and promise we never will"</p>



<a name="232835428"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835428" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835428">(Apr 01 2021 at 22:44)</a>:</h4>
<p>We absolutely could, and that's not necessarily a bad idea.</p>



<a name="232835469"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835469" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835469">(Apr 01 2021 at 22:45)</a>:</h4>
<p>At the very least, I would propose adding language like the following to the RFC:</p>
<p>Future versions of Rust may add new interpretations of <code>&lt;identifier&gt;#&lt;identifier&gt;</code> or <code>&lt;identifier&gt;"&lt;string contents&gt;"</code>. The addition of such new interpretations is not considered a breaking change. This space exists for future use by the compiler, and a program using tokens in this space that are not defined by the stable Rust compiler (including via a macro) is not covered by Rust's usual guarantees of stability and forward-compatibility.</p>



<a name="232835496"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835496" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835496">(Apr 01 2021 at 22:45)</a>:</h4>
<p>though I would want to see some motivation from macro authors to know if this would actually be useful, currently I have absolutely no idea what a real-world use case would look like</p>



<a name="232835576"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835576" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835576">(Apr 01 2021 at 22:46)</a>:</h4>
<p><span class="user-mention silent" data-user-id="256342">bstrie</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/232835496">said</a>:</p>
<blockquote>
<p>though I would want to see some motivation from macro authors to know if this would actually be useful, currently I have absolutely no idea what a real-world use case would look like</p>
</blockquote>
<p>Sure. I don't think <em>this</em> RFC should do anything in that area. Reserving the space means we <em>could</em> choose to assign some of it for such a purpose in the future.</p>



<a name="232835630"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835630" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835630">(Apr 01 2021 at 22:47)</a>:</h4>
<p>Any objections to the above language or something similar to it? Right now the RFC doesn't clearly define what it means to "reserve" the syntax, other than making it parse as one token.</p>



<a name="232835735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232835735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232835735">(Apr 01 2021 at 22:48)</a>:</h4>
<p>(I stopped using a <code> ``` </code> block for that, because it prevented wrapping and made it a pain to read.)</p>



<a name="232837027"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232837027" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232837027">(Apr 01 2021 at 23:02)</a>:</h4>
<p><span class="user-mention" data-user-id="256342">@bstrie</span> I posted a few suggestions regarding language.</p>



<a name="232837078"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232837078" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232837078">(Apr 01 2021 at 23:02)</a>:</h4>
<p>Those suggestions aside, I think this looks good, and I'd be thrilled to propose it for merge so that we can have it available for edition planning in a timely fashion.</p>



<a name="232837711"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232837711" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232837711">(Apr 01 2021 at 23:08)</a>:</h4>
<p><span class="user-mention silent" data-user-id="256342">bstrie</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/232835496">said</a>:</p>
<blockquote>
<p>though I would want to see some motivation from macro authors to know if this would actually be useful, currently I have absolutely no idea what a real-world use case would look like</p>
</blockquote>
<p>hey <span class="user-mention" data-user-id="232018">@Daniel Henry-Mantilla</span> <span aria-label="wink" class="emoji emoji-1f609" role="img" title="wink">:wink:</span></p>



<a name="232838993"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232838993" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Daniel Henry-Mantilla <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232838993">(Apr 01 2021 at 23:23)</a>:</h4>
<p><span aria-label="sweat smile" class="emoji emoji-1f605" role="img" title="sweat smile">:sweat_smile:</span> I personally can't think of any motivation for such special tokenizations, except for wanting to polyfill the very feature that would break it <span aria-label="big smile" class="emoji emoji-1f604" role="img" title="big smile">:big_smile:</span>. To be fair, it would have to be a very <em>specific</em> DSL of sorts to ever need <code>k#&lt;ident&gt;</code> <em>without whitespace in between</em> to work (I agree with the comparison with <code>$(,)?</code> in that regard: this is breakage that in practice shouldn't be observed; and I am very curious about that crater run).</p>
<p>As an aside, I think that macro authors know (or ought to) that they depend on Rust's lexer to operate, and that this lexer's primary purpose is obviously for the language; I'd hope that for those needing special lexing-related requirements (<em>e.g.</em>, when whitespace is semantically meaningful) they know they have the option to bundle the DSL contents / macro inout within a string literal, (or within an external file if processing it from within a <code>build.rs</code>) or the currently hacky (but which could be blessed not to be) approach of using proc-macro + fs APIs to get the source code fed to the macro (the hacky part there being the impure fs access, so the fix would thus be to offer some kind of <code>Span::source_code(&amp;Span) -&gt; String</code> API). <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<a name="232840644"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232840644" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232840644">(Apr 01 2021 at 23:43)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/232834647">said</a>:</p>
<blockquote>
<p>Is there any reasonable way we could <em>prevent</em> macros or similar from using such tokens if they don't actually exist in the language?</p>
</blockquote>
<p>Trying to use <code>r#$foo</code> in a macro argument is a tokenization error, so I think we could do that?</p>



<a name="232879197"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232879197" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232879197">(Apr 02 2021 at 09:29)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/232834647">said</a>:</p>
<blockquote>
<p>Is there any reasonable way we could <em>prevent</em> macros or similar from using such tokens if they don't actually exist in the language?</p>
</blockquote>
<p>is that something we should prevent? people can already use any keyword we have in their macro for anything, right? plenty of macros match specifically on <code>let</code> and <code>for</code>, why not on <code>k#yeet</code>?</p>



<a name="232883653"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232883653" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232883653">(Apr 02 2021 at 10:23)</a>:</h4>
<p><span class="user-mention" data-user-id="310399">@Mara</span> That wasn't my concern. I was worried about macros using keywords we <em>don't</em> have, or reserved syntax we don't assign a meaning to yet.</p>



<a name="232883840"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232883840" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232883840">(Apr 02 2021 at 10:25)</a>:</h4>
<p>in which cases would that matter?</p>
<p>i suppose one could match on <code>($e:expr k#yeet)</code>, which will start failing to compile if we later decide <code>k#yeet</code> can be part of an <code>expr</code>. but for simple <code>:tt</code>'s it should be fine. and for every other matcher like <code>:expr</code> we could just forbid these type of tokens afterwards, just like we already do for regular identifiers</p>



<a name="232884033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232884033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232884033">(Apr 02 2021 at 10:27)</a>:</h4>
<p>but i don't see a problem with someone matching on <code>(k#yeet $e:expr)</code>, even if we'd at some point accept <code>k#yeet 1</code> as an <code>:expr</code>. doesn't really break anything. it might make their macro inconsistent with our meaning of <code>k#yeet</code>, but they can already to the exact same for any other existing or future keyword.</p>



<a name="232884103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232884103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232884103">(Apr 02 2021 at 10:28)</a>:</h4>
<p>It would matter if some macro started using it and that caused us to hesitate to define a keyword because a macro in the ecosystem uses it, which would defeat the purpose of reserving the space.</p>



<a name="232884144"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232884144" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232884144">(Apr 02 2021 at 10:29)</a>:</h4>
<p>That said, I've already argued that we <em>should</em> allow this, for the "prototype via macro" case.</p>



<a name="232884467"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232884467" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232884467">(Apr 02 2021 at 10:32)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/232884103">said</a>:</p>
<blockquote>
<p>It would matter if some macro started using it and that caused us to hesitate to define a keyword because a macro in the ecosystem uses it, which would defeat the purpose of reserving the space.</p>
</blockquote>
<p>but that's still the case for non-prefixed keywords, right?  if someone makes a macro that makes <code>yeet</code> work that gets very popular, we'd still be hesitant to add a <code>yeet</code> that works differently.</p>



<a name="232904704"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232904704" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232904704">(Apr 02 2021 at 14:26)</a>:</h4>
<p>Couldn't "prototype via macro" use plain <code>yeet</code>? IMO, let the reserved space be truly reserved, not even a valid <code>tt</code>. That's what made <code>r#ident</code> possible.</p>



<a name="232930161"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232930161" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232930161">(Apr 02 2021 at 17:49)</a>:</h4>
<p><span class="user-mention" data-user-id="138448">@cuviper</span> You know, I hadn't even thought of that?</p>



<a name="232930173"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232930173" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232930173">(Apr 02 2021 at 17:50)</a>:</h4>
<p>That's a great point.</p>



<a name="232930420"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232930420" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232930420">(Apr 02 2021 at 17:51)</a>:</h4>
<p>That's a very good argument for erroring at lex time on any reserved syntax Rust doesn't specifically support.</p>



<a name="232930488"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/232930488" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#232930488">(Apr 02 2021 at 17:52)</a>:</h4>
<p><span class="user-mention" data-user-id="256342">@bstrie</span> ^</p>



<a name="233250275"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233250275" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233250275">(Apr 06 2021 at 00:34)</a>:</h4>
<p>I'm working on changes to the RFC now. We want to make these into lex errors, yes? Because I'm interpreting that as though <code>foo!(r#bar)</code> ought to fail to lex, but <code>r#bar</code> is valid Rust <em>outside</em> of macros today. Should I not just worry about covering that case?</p>



<a name="233250678"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233250678" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233250678">(Apr 06 2021 at 00:40)</a>:</h4>
<p>I'd say it's natural to leave that up to future work on revising the raw identifier RFC, if desired</p>



<a name="233260710"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233260710" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233260710">(Apr 06 2021 at 03:09)</a>:</h4>
<p>No, <code>r#bar</code> in a macro should be a single <code>:ident</code> token, but <code>z#bar</code> should be a lex error -- at least until we give that some meaning, and then it should do that new thing.</p>



<a name="233313302"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233313302" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233313302">(Apr 06 2021 at 12:57)</a>:</h4>
<p>if we make them lexing errors, that means we don't just break (proc) macros that use a # or "" prefix in their syntax, but we also make it impossible for them to fix it. a proc macro that accepts <code>(f"asdf")</code> right now would just stop working entirely, instead of just passsing <code>f"asdf"</code> as one token instead of two (which the proc macro author can still work with).</p>



<a name="233349022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233349022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233349022">(Apr 06 2021 at 16:23)</a>:</h4>
<p><span class="user-mention" data-user-id="310399">@Mara</span> that's a good point too. They do have one fix available, to space out their tokens (<code>f "asdf"</code>), but I can see how that's distasteful. I guess if we do want to let this be lexed, it needs to be in such a way that it won't break when the language gives that same syntax a real meaning.</p>



<a name="233349347"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233349347" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233349347">(Apr 06 2021 at 16:25)</a>:</h4>
<p>so if we make it one token, it needs to be one token forevermore</p>



<a name="233349415"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233349415" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233349415">(Apr 06 2021 at 16:25)</a>:</h4>
<p>And reserved tokens should probably only be allowed as a raw <code>:tt</code>, and possibly not allowed in the follow sets of other macro types</p>



<a name="233351741"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233351741" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233351741">(Apr 06 2021 at 16:41)</a>:</h4>
<p>(btw the space to make it lex again is still visible in .source_text() and in the span location, so that would break hacky macros like <code>inline-python</code>)</p>



<a name="233394875"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233394875" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233394875">(Apr 06 2021 at 21:36)</a>:</h4>
<p><span class="user-mention silent" data-user-id="310399">Mara</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233313302">said</a>:</p>
<blockquote>
<p>if we make them lexing errors, that means we don't just break (proc) macros that use a # or "" prefix in their syntax, but we also make it impossible for them to fix it. a proc macro that accepts <code>(f"asdf")</code> right now would just stop working entirely, instead of just passsing <code>f"asdf"</code> as one token instead of two (which the proc macro author can still work with).</p>
</blockquote>
<p>"impossible to fix" seems a bit harsh, given that rustfix seems capable of automatically turning that into <code>f "asdf"</code>. I don't find that be particularly distasteful, but note that anyone who doesn't want to employ whitespace would still be capable of using literal <em>suffixes</em> here, which are already more-or-less reserved for use by macros: <a href="https://doc.rust-lang.org/reference/tokens.html#suffixes">https://doc.rust-lang.org/reference/tokens.html#suffixes</a> . So another possible migration would be <code>f"asdf"</code> -&gt; <code>"asdf"f</code>.</p>



<a name="233395725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233395725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233395725">(Apr 06 2021 at 21:43)</a>:</h4>
<p>WRT to macros that find hacky ways to observe significant whitespace, I have to assume that the developers understand that's an unsupported use of the macro system. it feels analogous to saying that proc_macro::TokenStream isn't powerful enough so you code directly against the rustc AST; that's deliberately eschewing stable abstractions in favor of unstable ones, so caveat emptor. anyone who wants to losslessly embed another language in macros needs to be using strings; even the <code>asm!</code> macro doesn't get a free pass here.</p>



<a name="233396770"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233396770" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233396770">(Apr 06 2021 at 21:53)</a>:</h4>
<p>huh, I had no idea that arbitrary suffixes are already permitted</p>



<a name="233397626"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233397626" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233397626">(Apr 06 2021 at 22:00)</a>:</h4>
<p>I didn't until I started writing this RFC, and I'm pretty sure almost nobody else does too; that's why I link it multiple times in my RFC :P</p>



<a name="233397715"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233397715" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233397715">(Apr 06 2021 at 22:01)</a>:</h4>
<p>Thanks for working on this, @bstrie.</p>



<a name="233436712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233436712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233436712">(Apr 07 2021 at 06:11)</a>:</h4>
<blockquote>
<p>"impossible to fix" seems a bit harsh, given that rustfix seems capable of automatically turning that into <code>f "asdf"</code></p>
</blockquote>
<p>I have several proc macros for which that's a breaking change, where <code>f "asdf"</code> is not the same as <code>f"asdf"</code>. and it's not weird at all to expect macros that would make things like <code>c"asdf"</code> or whatever work as a 'polyfill'. for keywords like <code>k#yeet</code> the argument was that macros could just use <code>yeet</code> itself. but for these we're not using the prefix as a temporary placeholder until an edition change. so that argument doesn't hold here.</p>



<a name="233436814"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233436814" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233436814">(Apr 07 2021 at 06:12)</a>:</h4>
<p>and even for macros that aren't looking at whitespace, there's a ton of macros that implement some DSL, possibly containing things like <code>f"asdf"</code> or <code>c"asdf"</code> etc. adding spaces there even though that's not the syntax in the language they're implementing/imitating is far from great.</p>



<a name="233479791"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233479791" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Daniel Henry-Mantilla <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233479791">(Apr 07 2021 at 12:43)</a>:</h4>
<p>Basic question here: why do <code>&lt;prefix&gt;#&lt;ident&gt;</code> and <code>&lt;prefix&gt;"…"</code> &amp; <code>&lt;prefix&gt;'…'</code> have to feature the same lexing behavior? To me, the latter (pair) is an extended way to feature "enhanced literals", this time using prefixes rather than suffixes. These literals, should they be given meaning, would expand to stand-alone expressions. This means that there is no problem in letting them be tokenized as <code>:tt</code>s  that are also compatible with <code>:expr</code> captures (and no other high-level capture).</p>
<p>The problem lies in <code>&lt;prefix&gt;#&lt;ident&gt;</code>. If these are given, for instance, raw keyword meaning, they could become valid <code>:expr</code> (or any other capture) prefixes (say, <code>k#trust_me</code>), but they could also become invalid <code>:expr</code> prefixes (say <code>k#typeof</code>). This is what could be breaking in the future, and why it should cause compilation errors when trying to match these against such captures (≠ fallbacking to the following rule!). Since such a behavior may seem complex to express understand, the simplified "lexing error" suffices to ensure these properties.</p>
<p>Thus, there is legitimate motivation to make <code>&lt;prefix&gt;#&lt;ident&gt;</code> be a lexing error while unassigned, but I fail to see such motivation also apply to / extend to prefixed quoted literals, similar to how there is no lexing error when using suffixed (quoted) literals: has anyone an example of some prefixed quoted literal that may end up not having <code>:expr</code> semantics only? (maybe with also <code>:literal</code>)</p>



<a name="233523793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233523793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233523793">(Apr 07 2021 at 16:48)</a>:</h4>
<p>Sure, I can easily imagine some uses that aren't all the same token type. <code>rust_ident"with-non-ident-chars"</code>.</p>



<a name="233523879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233523879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233523879">(Apr 07 2021 at 16:49)</a>:</h4>
<p>(not advocating, just hypothesizing)</p>



<a name="233524284"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233524284" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233524284">(Apr 07 2021 at 16:51)</a>:</h4>
<p>Quasiquoters. Compile-time <code>format!</code> to produce an ident.</p>



<a name="233524453"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233524453" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233524453">(Apr 07 2021 at 16:52)</a>:</h4>
<p>Much of the point of reserving these areas of syntax is to let us define them arbitrarily in the future, without worrying about any conflicts with existing code, including macros.</p>



<a name="233524537"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233524537" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233524537">(Apr 07 2021 at 16:53)</a>:</h4>
<p>So by erroring in tokenization if we see something we don't understand, we prevent a macro from using that area of syntax in a way that will conflict with rust's own future usage.</p>



<a name="233534538"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233534538" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233534538">(Apr 07 2021 at 17:59)</a>:</h4>
<p>One of the things that had me excited for the <code>ident"string"</code> form was that I've always thought it odd that <code>a"xyz"</code> and <code>b"xyz"</code> tokenize differently.  I think I could get behind the idea that we say those tokenize as one token and will always be <code>:literal</code>s -- but that if they actually expand into a use of use a token in rust then they error if not yet given a defined meaning.</p>
<p>Though that argument only applies to the string-literal format, where I think the precedent of how we'd use it is pretty strong already.  Something like <code>k#yolo</code> I think the tokenization error is probably best.</p>



<a name="233573565"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233573565" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233573565">(Apr 07 2021 at 22:34)</a>:</h4>
<p>I'm happy to revise the RFC to say that <code>foo#bar</code> is a tokenization error whereas <code>foo"bar"</code>, <code>foo'x'</code>, and <code>foo#123</code> will produce a single token but otherwise fail to parse, although if people think that they should match something other than <code>:tt</code> (<code>:expr</code>, etc... I've never heard of <code>:literal</code> before) then speak up with an exhaustive list, because I have no opinion there.</p>



<a name="233580228"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233580228" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233580228">(Apr 07 2021 at 23:43)</a>:</h4>
<p>I personally still think all four should produce a tokenization error if they're not yet defined in Rust syntax.</p>



<a name="233580257"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233580257" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233580257">(Apr 07 2021 at 23:43)</a>:</h4>
<p>That said, I'd <em>also</em> be fine with only doing that tokenization-time-error on Rust 2021, if doing so would break 2018 and 2015 and we could otherwise make this change.</p>



<a name="233583323"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233583323" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233583323">(Apr 08 2021 at 00:17)</a>:</h4>
<p>I'm also fine with it being a tokenization error in this RFC, since we can always loosen things later if we want.</p>



<a name="233786335"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233786335" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233786335">(Apr 09 2021 at 08:37)</a>:</h4>
<p>it seemed simple at first, but the prefixed string literals are pretty tricky. <code>r"\"</code> is a valid token/literal. <code>"\"</code> is not. how about <code>fr"\"</code>? regardless of whether we want that to give an error or not because of the prefix, we still need to know if we should interpret that second <code>"</code> as the end or as an escaped <code>"</code> inside the literal. otherwise we can't even continue tokenizing.</p>



<a name="233842457"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233842457" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Julian-Wollersberger <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233842457">(Apr 09 2021 at 15:24)</a>:</h4>
<p>I've made a PR for a crater run: <a href="https://github.com/rust-lang/rust/pull/84037">https://github.com/rust-lang/rust/pull/84037</a></p>



<a name="233842748"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233842748" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Julian-Wollersberger <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233842748">(Apr 09 2021 at 15:26)</a>:</h4>
<p>I took some shortcuts while implementing that, like it errors on all editions and also considers <code>ident#</code> followed by _anything_ illegal. But should be good enough for crater</p>



<a name="233843089"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233843089" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Julian-Wollersberger <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233843089">(Apr 09 2021 at 15:28)</a>:</h4>
<p><span class="user-mention silent" data-user-id="256342">bstrie</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233573565">said</a>:</p>
<blockquote>
<p>I'm happy to revise the RFC to say that <code>foo#bar</code> is a tokenization error whereas <code>foo"bar"</code>, <code>foo'x'</code>, and <code>foo#123</code> will produce a single token but otherwise fail to parse, </p>
</blockquote>
<p>If it fails to parse, it should be irrelevant how many tokens it produces. (Aside from error recovery)</p>



<a name="233845789"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233845789" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233845789">(Apr 09 2021 at 15:44)</a>:</h4>
<blockquote>
<p>If it fails to parse, it should be irrelevant how many tokens it produces.</p>
</blockquote>
<p>no, this is relevant for macros. you could still do <code>a!(foo"bar")</code> even if <code>foo"abc"</code> never parses as (part of) an expression/item/anything.</p>



<a name="233845891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233845891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233845891">(Apr 09 2021 at 15:45)</a>:</h4>
<p>(assuming parsing here means the tokens-&gt;ast conversion, not file-&gt;tokens)</p>



<a name="233869366"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233869366" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233869366">(Apr 09 2021 at 18:27)</a>:</h4>
<p><span class="user-mention silent" data-user-id="310399">Mara</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233786335">said</a>:</p>
<blockquote>
<p>it seemed simple at first, but the prefixed string literals are pretty tricky. <code>r"\"</code> is a valid token/literal. <code>"\"</code> is not. how about <code>fr"\"</code>? regardless of whether we want that to give an error or not because of the prefix, we still need to know if we should interpret that second <code>"</code> as the end or as an escaped <code>"</code> inside the literal. otherwise we can't even continue tokenizing.</p>
</blockquote>
<p>I agree that this is an interesting problem. But we know we'd be giving a hard error either way, so this is more a quality-of-implementation issue regarding our ability to continue on and give more errors, rather than a correctness issue.</p>



<a name="233869523"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233869523" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233869523">(Apr 09 2021 at 18:28)</a>:</h4>
<p>And there are heuristics we could <em>potentially</em> use if we think the error messages are too awful. "If prefix identifier has an 'r' in it, and we failed to tokenize, and we have a <code>\"</code>..."</p>



<a name="233869576"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233869576" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233869576">(Apr 09 2021 at 18:29)</a>:</h4>
<p>But to a first approximation, this is "how good is the error message for code that shouldn't successfully lex".</p>



<a name="233884379"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233884379" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233884379">(Apr 09 2021 at 20:17)</a>:</h4>
<p>i'm no longer convinced we should reserve all prefixes for string literals, since we can't even tokenize those properly if we don't know what the prefix means for the escaping. i originally wanted to just make <code>f".."</code> parse as one token, such that we could turn it into a format_args!() shortcut later if we want to. that could still tokenize just fine. (maybe it'd also have <code>fr".."</code> just like <code>br".."</code>) but erroring on <code>xyz".."</code> for no specific reason while breaking any proc macros that use that kind of syntax doesn't seem like a good idea.</p>



<a name="233889364"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233889364" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233889364">(Apr 09 2021 at 21:00)</a>:</h4>
<p>Hmmm. We could reserve [a-z]"...", on the assumption that nothing else will tokenize differently other than <code>r</code>.</p>



<a name="233979513"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233979513" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233979513">(Apr 10 2021 at 19:13)</a>:</h4>
<p>Is that a safe assumption? Suppose we had format literals like <code>f"2 + 2 = {2 + 2}"</code>. Would that be one token?</p>



<a name="233979589"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233979589" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233979589">(Apr 10 2021 at 19:14)</a>:</h4>
<p>If a token can contain zero or more (parsed!) expressions, it's really stretching the definition of what a token is</p>



<a name="233979780"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233979780" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233979780">(Apr 10 2021 at 19:17)</a>:</h4>
<p>and to be clear, the reason why a non-format-string aware tokenization could get confused is because of examples where unescaped strings appear in the expression, like <code>f"a = {String::new("a")}"</code></p>



<a name="233979879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233979879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233979879">(Apr 10 2021 at 19:18)</a>:</h4>
<p>Can you elaborate on what the tokenizing problem is? Here's what I have as the grammar for the string literal reservation:</p>
<blockquote>
<p>RESERVED_STRING_LITERAL : IDENTIFIER_OR_KEYWORD RAW_BYTE_STRING_LITERAL | IDENTIFIER_OR_KEYWORD BYTE_STRING_LITERAL | IDENTIFIER_OR_KEYWORD&lt;sub&gt;Except <code>b</code>&lt;/sub&gt; RAW_STRING_LITERAL | IDENTIFIER_OR_KEYWORD&lt;sub&gt;Except <code>b</code>, <code>r</code>, <code>br</code>&lt;/sub&gt; STRING_LITERAL</p>
</blockquote>
<p>Now my BNF is rusty so let me know if that looks bogus to anyone (I'm lifting the curious "Except" clauses from elsewhere in the reference, so at least there's precedence). But it hopefully answers your questions: <code>fr"\"</code> would be reserved as an identifier <code>f</code> prefixing a raw string literal <code>r"\"</code>; <code>x"\"</code> would not  be reserved, at least not until it encounters a closing <code>"</code>, and if it doesn't ever encounter a closing <code>"</code> then your macro already doesn't compile today.</p>
<p>So yes, this RFC implicitly assumes that to make a raw string you will need to <em>end</em> your identifier in <code>r</code>, or else you will have a non-raw string. I think this is acceptable; the precedence of <code>b</code> combining gracefully with <code>r</code> suggest to me that we'd want such reasonable intermixing as ought to be allowed, and furthermore that <code>r</code> <em>should</em> always be the last character in the prefix if you want a raw string, due to the optional <code>#</code>s.</p>



<a name="233979986"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/233979986" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#233979986">(Apr 10 2021 at 19:20)</a>:</h4>
<p>I would expect format strings <code>f"blah"</code> to parse as strings, and <code>fr"blah"</code> to parse as raw strings</p>



<a name="234000320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234000320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Daniel Henry-Mantilla <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234000320">(Apr 11 2021 at 00:08)</a>:</h4>
<p><span class="user-mention silent" data-user-id="271719">Mario Carneiro</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233979780">said</a>:</p>
<blockquote>
<p>and to be clear, the reason why a non-format-string aware tokenization could get confused is because of examples where unescaped strings appear in the expression, like <code>f"a = {String::new("a")}"</code></p>
</blockquote>
<p>I think that under the current proposal that example wouldn't lex; it would have to be written as <code>f"a = { String::from(\"a\") }"</code> or <code>fr#"a = { String::from("a") }"#</code></p>
<p><span class="user-mention silent" data-user-id="256342">bstrie</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233979879">said</a>:</p>
<blockquote>
<p>So yes, this RFC implicitly assumes that to make a raw string you will need to <em>end</em> your identifier in <code>r</code>, or else you will have a non-raw string. I think this is acceptable</p>
</blockquote>
<p>I think it is acceptable too, but it should still be noted as a limitation: words that end in <code>r</code> will be confusing to use and will thus have to be avoided: <code>bar"\n"</code></p>
<p><span class="user-mention silent" data-user-id="271719">Mario Carneiro</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233979780">said</a>:</p>
<blockquote>
<p>If a token can contain zero or more (parsed!) expressions, it's really stretching the definition of what a token is</p>
</blockquote>
<p>If we are talking about the token <em>tree</em> that a macro can see, then there is already precedence for <em>parsed</em> AST nodes such as <code>1 + 1</code> to appear as a single token tree to (nested) macro( invocation)s when the node ends up captured within a higher-level metavariable, such as <code>:expr</code>. That's the magic of invisible delimiters.</p>
<p>So, yes, things like <code>f"Hello, {world}!"</code> could be lexed as a single token (tree) even if such token tree ends up being <code>⌈ ::std::format!("Hello, {world}!", world = world) ⌉</code> where I am using <code>⌈</code> and <code>⌉</code> to denote the invisible delimiters.</p>



<a name="234004891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234004891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234004891">(Apr 11 2021 at 01:31)</a>:</h4>
<p>I'll add a note about these implicit assumptions in the RFC.</p>



<a name="234207933"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234207933" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234207933">(Apr 12 2021 at 18:31)</a>:</h4>
<p>Hey <span class="user-mention" data-user-id="125270">@scottmcm</span> ---</p>



<a name="234207961"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234207961" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234207961">(Apr 12 2021 at 18:31)</a>:</h4>
<p>we were talking about reserved prefixes for idents and literals in the edition 2021 planning meeting today</p>



<a name="234207973"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234207973" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234207973">(Apr 12 2021 at 18:32)</a>:</h4>
<p>it seems like one of the agenda items that is most 'at risk'</p>



<a name="234208051"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234208051" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234208051">(Apr 12 2021 at 18:32)</a>:</h4>
<p>I was thinking it'd be nice to have a summary of the current state and what is being considered</p>



<a name="234208080"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234208080" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234208080">(Apr 12 2021 at 18:32)</a>:</h4>
<p>I'm going to whip up a hackmd</p>



<a name="234208104"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234208104" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234208104">(Apr 12 2021 at 18:32)</a>:</h4>
<p>also cc <span class="user-mention" data-user-id="256342">@bstrie</span> of course, sorry</p>



<a name="234208280"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234208280" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234208280">(Apr 12 2021 at 18:33)</a>:</h4>
<p>The RFC only needs one more checkbox: <a href="https://github.com/rust-lang/rfcs/pull/3101">https://github.com/rust-lang/rfcs/pull/3101</a>   But I do have some concerns, see also the comment there and the discussion in the zulip thread here.</p>



<a name="234208664"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234208664" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234208664">(Apr 12 2021 at 18:36)</a>:</h4>
<p>I <a href="https://hackmd.io/gK4dEyh5TgmnVfFyJL6e5Q">created this hackmd</a></p>



<a name="234208716"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234208716" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234208716">(Apr 12 2021 at 18:36)</a>:</h4>
<p><span class="user-mention" data-user-id="310399">@Mara</span> it has a few questions for you</p>



<a name="234209434"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209434" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209434">(Apr 12 2021 at 18:41)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116009">nikomatsakis</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/234208051">said</a>:</p>
<blockquote>
<p>I was thinking it'd be nice to have a summary of the current state and what is being considered</p>
</blockquote>
<p>The only option being considered at this time is whether encountering a reserved prefix should result in a pre-tokenization error or a post-tokenization error, i.e. whether macros should be able to consume the prefixed item (but not use it literally).</p>
<p>There is one thing that I've been meaning to make more explicit in the text: that any string prefix ending in <code>r</code> is tokenized as a raw string would be, whereas any string prefix not ending in <code>r</code> is tokenized as a non-raw string would be. Currently it just implicitly assumes that new string prefixes will want "compose" in the same manner as <code>b</code> and <code>r</code> (so a hyptothetical format string might be written <code>f""</code>, with <code>fr""</code> for the raw version), even though it reserves the full space of literals, which could throw a wrench in, for example, <code>november"foo"</code> tokenizing differenly from <code>july"foo"</code>.</p>



<a name="234209623"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209623" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209623">(Apr 12 2021 at 18:42)</a>:</h4>
<p>I'd be in favor of that</p>



<a name="234209641"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209641" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209641">(Apr 12 2021 at 18:42)</a>:</h4>
<p>I should add that as a variant</p>



<a name="234209726"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209726" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209726">(Apr 12 2021 at 18:43)</a>:</h4>
<p>ah, one catch</p>



<a name="234209736"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209736" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209736">(Apr 12 2021 at 18:43)</a>:</h4>
<p>I think byte strings also have some conditions</p>



<a name="234209754"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209754" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209754">(Apr 12 2021 at 18:43)</a>:</h4>
<p>I would be ok with moving those out from the tokenizer to the parser though, personally</p>



<a name="234209923"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209923" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209923">(Apr 12 2021 at 18:44)</a>:</h4>
<p><span class="user-mention" data-user-id="256342">@bstrie</span> take a look at <a href="https://hackmd.io/gK4dEyh5TgmnVfFyJL6e5Q">the hackmd</a> and let me know if anything is missing/misleading from your pov</p>



<a name="234209984"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234209984" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234209984">(Apr 12 2021 at 18:45)</a>:</h4>
<p>Also cc <span class="user-mention" data-user-id="127859">@Taylor Cramer</span> <span class="user-mention" data-user-id="116083">@pnkfelix</span> and <span class="user-mention" data-user-id="239881">@Josh Triplett</span> about <a href="https://github.com/rust-lang/rfcs/pull/3101">https://github.com/rust-lang/rfcs/pull/3101</a></p>



<a name="234215240"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215240" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215240">(Apr 12 2021 at 19:22)</a>:</h4>
<p>Oh, interesting.  I would have thought it was already a parser error, but it does look like it's a tokenization-time error now <a href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=b53f34c2e811e51d0faaf2a1257d9405">https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=b53f34c2e811e51d0faaf2a1257d9405</a></p>



<a name="234215529"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215529" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215529">(Apr 12 2021 at 19:24)</a>:</h4>
<p>I want to raise one other question</p>



<a name="234215569"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215569" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215569">(Apr 12 2021 at 19:24)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> was concerned because he doesn't want us to get in the situation where we have stated that we reserve this syntax, but because it is in use by popular procedural macros we are reluctant to claim it</p>



<a name="234215597"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215597" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215597">(Apr 12 2021 at 19:24)</a>:</h4>
<p>I think that's a reasonable concern, and tokenization errors help to prevent it</p>



<a name="234215623"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215623" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215623">(Apr 12 2021 at 19:25)</a>:</h4>
<p>however, I think it's also true that so long as macro-rules doesn't really accept the form</p>



<a name="234215635"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215635" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215635">(Apr 12 2021 at 19:25)</a>:</h4>
<p>we don't have to worry that much</p>



<a name="234215680"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215680" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215680">(Apr 12 2021 at 19:25)</a>:</h4>
<p>e.g., if I am writing a procedural macro, and I am using (say) <code>syn</code> to parse rust code, it will not accept the new forms until I upgrade to a newer syn</p>



<a name="234215709"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215709" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215709">(Apr 12 2021 at 19:25)</a>:</h4>
<p>so existing code using said DSL will continue to work even if we've claimed the syntax, i believe</p>



<a name="234215768"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215768" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215768">(Apr 12 2021 at 19:26)</a>:</h4>
<p>(the main danger I can see would be that someone <em>attempts</em> to parse the code as an expression, or whatever, and expects that parsing to fail on the new syntax so they can apply their fallback)</p>



<a name="234215790"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234215790" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234215790">(Apr 12 2021 at 19:26)</a>:</h4>
<p>am I missing something?</p>



<a name="234894192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234894192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234894192">(Apr 16 2021 at 17:44)</a>:</h4>
<p>So, in the interests of getting this reservation into the edition...</p>



<a name="234894254"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234894254" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234894254">(Apr 16 2021 at 17:45)</a>:</h4>
<p>Perhaps we should just <em>drop</em> the reservation of string constants, for the time being?</p>



<a name="234894297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234894297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234894297">(Apr 16 2021 at 17:45)</a>:</h4>
<p>Let's get an RFC accepted for <code>k#keyword</code>, in a timely fashion, and continue considering strings in parallel.</p>



<a name="234894312"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234894312" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234894312">(Apr 16 2021 at 17:45)</a>:</h4>
<p>I don't want to block the former on resolution of the latter.</p>



<a name="234894709"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234894709" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234894709">(Apr 16 2021 at 17:48)</a>:</h4>
<p>Does that mean it should also drop <code>foo'a'</code>?  And what about <code>foo#123</code>?</p>
<p>My instinct is to keep the number version but not the character or string version, but I haven't thought that through in depth.</p>



<a name="234894996"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234894996" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234894996">(Apr 16 2021 at 17:50)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> i think it's already resolved? the rfc is just waiting for fcp now.</p>



<a name="234895251"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234895251" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234895251">(Apr 16 2021 at 17:52)</a>:</h4>
<p>Ah! Re-checking now.</p>



<a name="234895490"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234895490" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234895490">(Apr 16 2021 at 17:54)</a>:</h4>
<p>it now considers <code>bar".."</code> a raw string literal, just like <code>br".."</code>. and like niko said we can move any validation that are specific to byte strings or other types of string (escape sequences like <code>\x..</code> or <code>\u..</code> or non-ascii characters) from the lexer to the parser.</p>



<a name="234895739"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234895739" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234895739">(Apr 16 2021 at 17:55)</a>:</h4>
<p>...that is not how I would have expected that to be resolved.</p>



<a name="234895895"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234895895" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234895895">(Apr 16 2021 at 17:56)</a>:</h4>
<p>I personally would have expected to see it resolved by just saying "if we see a string prefix we don't recognize, we'll make a best effort to proceed, but the most we can guarantee is an error on that unknown prefix".</p>



<a name="234895978"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234895978" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234895978">(Apr 16 2021 at 17:57)</a>:</h4>
<p>what about hash-bracketed string literals?  <code>foo####"..."####</code></p>



<a name="234896207"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234896207" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234896207">(Apr 16 2021 at 17:59)</a>:</h4>
<p><span class="user-mention" data-user-id="138448">@cuviper</span> <code>keyword#"</code> should probably be reserved as well. But I really don't want to turn this into a katamari of all possible syntax reservations and prevent it from making progress.</p>



<a name="234896360"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234896360" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234896360">(Apr 16 2021 at 18:00)</a>:</h4>
<p>understood</p>



<a name="234896620"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234896620" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234896620">(Apr 16 2021 at 18:01)</a>:</h4>
<p><span class="user-mention silent" data-user-id="138448">cuviper</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/234895978">said</a>:</p>
<blockquote>
<p>what about hash-bracketed string literals?  <code>foo####"..."####</code></p>
</blockquote>
<p>that needs an <code>r</code>. (<code>bar".."</code> is just a prefix of <code>ba</code> on a raw string literal.)</p>



<a name="234896712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234896712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234896712">(Apr 16 2021 at 18:01)</a>:</h4>
<blockquote>
<p>we'll make a best effort to proceed</p>
</blockquote>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> but then (proc) macros can't rely on these things tokenizing, which was the point of the change</p>



<a name="234897082"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897082" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897082">(Apr 16 2021 at 18:03)</a>:</h4>
<p>ah, I didn't realize the <code>r</code> was effectively special at the end of any prefix</p>



<a name="234897159"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897159" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897159">(Apr 16 2021 at 18:04)</a>:</h4>
<p>yeah we have <code>br"..."</code> but not <code>rb".."</code></p>



<a name="234897254"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897254" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897254">(Apr 16 2021 at 18:05)</a>:</h4>
<p>sure, but I thought of <code>br</code> as a unit, not <code>b</code>+<code>r"..."</code></p>



<a name="234897333"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897333" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897333">(Apr 16 2021 at 18:05)</a>:</h4>
<p><span class="user-mention silent" data-user-id="310399">Mara</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/234896712">said</a>:</p>
<blockquote>
<blockquote>
<p>we'll make a best effort to proceed</p>
</blockquote>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> but then (proc) macros can't rely on these things tokenizing, which was the point of the change</p>
</blockquote>
<p>I think I'm confused. I thought the notion was that we were just not going to commit to it being a <em>lexer</em> error, but that it was still going to be an error later in the compiler.</p>



<a name="234897483"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897483" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897483">(Apr 16 2021 at 18:06)</a>:</h4>
<p>it will still not parse. <code>bar".."</code> will give an error about an unknown prefix when parsed. but <code>some_macro! { bar".." }</code> can compile fine.</p>



<a name="234897511"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897511" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897511">(Apr 16 2021 at 18:06)</a>:</h4>
<p>(with <code>$_:tt</code>)</p>



<a name="234897527"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897527" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897527">(Apr 16 2021 at 18:06)</a>:</h4>
<p>(or <code>TokenStream</code>)</p>



<a name="234897909"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234897909" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234897909">(Apr 16 2021 at 18:09)</a>:</h4>
<p><span class="user-mention silent" data-user-id="138448">cuviper</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/234897254">said</a>:</p>
<blockquote>
<p>sure, but I thought of <code>br</code> as a unit, not <code>b</code>+<code>r"..."</code></p>
</blockquote>
<p>yeah right now both flags are used by the lexer (it rejects <code>b"å"</code> or <code>b"\u{1234}"</code> for example). but the checks for such escape codes and non-ascii characters can also happen in the parser, and then the only flag relevant for tokenizing is the final <code>r</code></p>



<a name="234898090"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898090" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898090">(Apr 16 2021 at 18:10)</a>:</h4>
<p>So, a macro can <em>eat</em> it, but can a macro actually <em>parse</em> it?</p>



<a name="234898124"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898124" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898124">(Apr 16 2021 at 18:11)</a>:</h4>
<p>proc macros only ever get tokens which they have to parse themselves..</p>



<a name="234898143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898143">(Apr 16 2021 at 18:11)</a>:</h4>
<p>and a regular macro can only match it as <code>:tt</code></p>



<a name="234898200"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898200" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898200">(Apr 16 2021 at 18:11)</a>:</h4>
<p>Would a regular macro be able to match it directly? <code>(k#foo a b c) =&gt; { ... handle k#foo }</code> ?</p>



<a name="234898342"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898342" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898342">(Apr 16 2021 at 18:12)</a>:</h4>
<p>I can potentially see value in a macro being able to eat-and-throw-away something unknown, if only so that we can have macros like <code>cfg_if!</code> throw away things unparsed (such as if the cfg controls use of a nightly-only feature).</p>



<a name="234898385"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898385" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898385">(Apr 16 2021 at 18:13)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/234898200">said</a>:</p>
<blockquote>
<p>Would a regular macro be able to match it directly? <code>(k#foo a b c) =&gt; { ... handle k#foo }</code> ?</p>
</blockquote>
<p>sure. just like any other token, yeah.</p>



<a name="234898936"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234898936" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234898936">(Apr 16 2021 at 18:16)</a>:</h4>
<p>You raised the objection on the thread; are you primarily concerned with existing macros that parse syntaxes like <code>ident"string"</code>?</p>



<a name="234899006"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234899006" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234899006">(Apr 16 2021 at 18:17)</a>:</h4>
<p>Because if that's the concern, I'd just as soon drop those from the reservation, if that means we can actually <em>reserve</em> <code>k#</code> in a "not even with a macro" way.</p>



<a name="234899079"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234899079" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234899079">(Apr 16 2021 at 18:17)</a>:</h4>
<p>I feel like this has gotten several things added to it, and then the added things turned out to generate conflicts, so we weakened the reservation mechanism to accommodate those added things...</p>



<a name="234899446"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234899446" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234899446">(Apr 16 2021 at 18:20)</a>:</h4>
<p>yes, mostly concerned about macros implementing some (DS)L with syntax like <code>f"..."</code> or <code>c"..."</code> or whatever. reserving <code>abc#xyz</code> is also a bit of a problem because of <code>quote !{}</code> using # for placeholders, but adding  a space there (through rustfix)  works.</p>



<a name="234975179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234975179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234975179">(Apr 17 2021 at 09:28)</a>:</h4>
<p>you know...</p>



<a name="234975193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234975193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234975193">(Apr 17 2021 at 09:28)</a>:</h4>
<p>I was wondering about <code>ident#"foo"#</code> too</p>



<a name="234975214"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234975214" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234975214">(Apr 17 2021 at 09:29)</a>:</h4>
<p>I had just sort of assumed we would parse that as a raw string or what have you, but I guess we could opt for lexer error</p>



<a name="234975225"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234975225" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234975225">(Apr 17 2021 at 09:29)</a>:</h4>
<p>it's not very obvious I suppose.</p>



<a name="234975232"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234975232" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234975232">(Apr 17 2021 at 09:29)</a>:</h4>
<p>I would expect that it would permit the use of <code>"</code> inside</p>



<a name="234975236"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/234975236" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#234975236">(Apr 17 2021 at 09:29)</a>:</h4>
<p>but I'm not sure if I would expect <code>\n</code> and friends to stop working</p>



<a name="235006110"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/235006110" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#235006110">(Apr 17 2021 at 17:56)</a>:</h4>
<p>I think it would be strange for any form of raw strings to not resemble <code>r"</code>. As I've mentioned, my mental model is that string literal prefixes are compositional.</p>



<a name="235006156"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/235006156" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#235006156">(Apr 17 2021 at 17:57)</a>:</h4>
<p>if people would like me to separate out each reservation into different RFCs, then I'm willing to do so. However, I don't quite understand what the theoretical objection is with the current draft RFC.</p>



<a name="235006259"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/235006259" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#235006259">(Apr 17 2021 at 17:59)</a>:</h4>
<p>another option is that if you want <code>ident#foo</code> to be a pre-tokenization error, and <code>ident"foo</code> to be a post-tokenization error, then that would also be possible. but to reiterate, I don't foresee any potential conflict between allowing <code>foo#bar</code> to <em>lex</em> now and making <code>foo#bar</code> <em>parse</em> later.</p>



<a name="235033622"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/235033622" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#235033622">(Apr 18 2021 at 01:33)</a>:</h4>
<p>I'm in favor of Niko's current proposal: we keep the reservation RFC as it was before, with lexer errors for unknown syntax, but we carve out the bits that existing crates use and allow them to tokenize and pass through to macros, and only error on those at parse time.</p>



<a name="236177704"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/236177704" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> XAMPPRocky <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#236177704">(Apr 26 2021 at 13:55)</a>:</h4>
<p><span class="user-mention silent" data-user-id="256342">bstrie</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/233397626">said</a>:</p>
<blockquote>
<p>I didn't until I started writing this RFC, and I'm pretty sure almost nobody else does too; that's why I link it multiple times in my RFC <span aria-label="stuck out tongue" class="emoji emoji-1f61b" role="img" title="stuck out tongue">:stuck_out_tongue:</span></p>
</blockquote>
<p>FWIW, I actually used this feature pretty recently in a macro to accept <code>1D</code>, <code>2D</code>, and <code>3D</code> as parameters. <span aria-label="smiley" class="emoji emoji-1f603" role="img" title="smiley">:smiley:</span></p>



<a name="238470099"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238470099" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238470099">(May 12 2021 at 13:09)</a>:</h4>
<p>Hi all. I'm interested in implementing this (if nobody has started earlier), but I have some questions. Hope here is the right place to ask.</p>
<ol>
<li>
<p>The RFC says <code>foo# bar</code> is three tokens. But currently <code>r# bar</code> failed with <code>found invalid character</code>. I think it is weird, isn't it? I wonder whether the error is intended or a fix is needed.</p>
</li>
<li>
<p>It seems the RFC does not describe some corner cases clearly. I give my expected results for them below, am I correct?</p>
</li>
</ol>
<ul>
<li>
<p><code>foo## bar</code><br>
    4 tokens <code>foo</code>, <code>#</code>, <code>#</code>, <code>bar</code> (NOTE: However,<code>r## bar</code> cannot compile now.)</p>
</li>
<li>
<p><code>foo#! bar</code><br>
    4 tokens <code>foo</code>,  <code>#</code>, <code>!</code>, <code>bar</code> (NOTE: However, <code>r#! bar</code> cannot compile now.)</p>
</li>
<li>
<p><code>foo##! bar</code><br>
    5 tokens <code>foo</code>, <code>#</code>, <code>#</code>, <code>!</code>, <code>bar</code> (NOTE: However, <code>r##! bar</code> cannot compile now.)</p>
</li>
<li>
<p><code>foo##" bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>foo##bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>r#foo#bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>r"foo"#bar</code><br>
    3 tokens <code>"foo"</code>, <code>#</code>, <code>bar</code></p>
</li>
</ul>
<p>Looking forward to and thanks for any feedback.</p>



<a name="238471865"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238471865" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238471865">(May 12 2021 at 13:20)</a>:</h4>
<ol>
<li>that's because <code>r#</code> now either starts a raw identifier or a raw string, so anything else after the <code>#</code> fails. that's fine, because the RFC tuns all unknown prefixes into errors.</li>
<li>it might be easier and more consistent if everything starting with <code>foo#</code> is an error. <code>r"foo"#bar</code> should stay three tokens, since we're only changing prefixes</li>
</ol>



<a name="238472109"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238472109" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238472109">(May 12 2021 at 13:21)</a>:</h4>
<p><span class="user-mention" data-user-id="407862">@lrh2000</span> You can use <a href="https://github.com/rust-lang/rust/issues/84599">the tracking issue</a> to claim the issue (say <code>@rustbot claim</code>, iirc). and that's also a good place to discuss details like these edge cases.</p>



<a name="238472250"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238472250" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238472250">(May 12 2021 at 13:22)</a>:</h4>
<p>(zulip is fine too for a 'live' discussion, but it's useful to keep track of design and implementation decisions like that in a github issue.)</p>



<a name="238475531"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238475531" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238475531">(May 12 2021 at 13:42)</a>:</h4>
<p><span class="user-mention silent" data-user-id="310399">Mara</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238471865">said</a>:</p>
<blockquote>
<ol>
<li>that's because <code>r#</code> now either starts a raw identifier or a raw string, so anything else after the <code>#</code> fails. that's fine, because the RFC tuns all unknown prefixes into errors.</li>
<li>it might be easier and more consistent if everything starting with <code>foo#</code> is an error. <code>r"foo"#bar</code> should stay three tokens, since we're only changing prefixes</li>
</ol>
</blockquote>
<ol>
<li>Yeah that's fine for now. But it is at least not necessary. What's more, I think it leads to bad consistency. For example, we may reserve  the prefix<code>xr#</code> and accept <code>xr# bar</code> as three tokens (that's what the RFC specified). However, once we decide to use <code>xr#</code> represents something like a raw string, <code>xr# bar</code> may not compile anymore (<code>found invalid character</code>).</li>
<li>Actually I think it is okay and easy to implement. However, the RFC specified we should accept <code>foo# bar</code> as three tokens. It actually complicates the problem a bit.</li>
</ol>



<a name="238475864"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238475864" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238475864">(May 12 2021 at 13:44)</a>:</h4>
<blockquote>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> You can use <a href="https://github.com/rust-lang/rust/issues/84599">the tracking issue</a> to claim the issue (say <code>@rustbot claim</code>, iirc). and that's also a good place to discuss details like these edge cases.</p>
</blockquote>
<p>OK, I'll do this later. (I didn't do this at first because I think I can get the feedback a little faster here.)</p>



<a name="238476139"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238476139" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238476139">(May 12 2021 at 13:46)</a>:</h4>
<p>it's not uncommon to slightly deviate from the original RFC if during implementation it turns out to make more sense that way. but those things should be discussed on the tracking issue to make sure everyone involved is aware, possibly nominating the issue for a discussion in a team meeeting</p>



<a name="238481863"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238481863" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238481863">(May 12 2021 at 14:23)</a>:</h4>
<blockquote>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> You can use <a href="https://github.com/rust-lang/rust/issues/84599">the tracking issue</a> to claim the issue (say <code>@rustbot claim</code>, iirc). and that's also a good place to discuss details like these edge cases.</p>
</blockquote>
<p>Done at <a href="https://github.com/rust-lang/rust/issues/84978">https://github.com/rust-lang/rust/issues/84978</a>. But it seems there are two tracking issues (why?) and I've found a different one.</p>



<a name="238512413"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238512413" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238512413">(May 12 2021 at 17:26)</a>:</h4>
<p>oh! we should probably mark the empty one as a duplicate or something :)</p>



<a name="238522904"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238522904" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238522904">(May 12 2021 at 18:35)</a>:</h4>
<blockquote>
<p>However, the RFC specified we should accept foo# bar as three tokens.</p>
</blockquote>
<p>as the author of the RFC, any "accept as three tokens" language should be read as "for the purposes of this RFC we don't actually care what the behavior is other than that it <em>won't</em> be a reserved prefix and therefore won't parse as one token; do whatever is most natural, most consistent, and least disruptive". I wasn't aware that <code>r# foo</code> didn't already parse as three tokens, for example; if I had know then I wouldn't have presumptively used the "three tokens" language. restrict the scope of the RFC as much as is reasonable.</p>



<a name="238523324"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238523324" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238523324">(May 12 2021 at 18:37)</a>:</h4>
<p>yeah these are typically the kind of details that only show up when starting the implementation</p>



<a name="238528191"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238528191" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238528191">(May 12 2021 at 19:08)</a>:</h4>
<p>We did talk about the possibility of reserving <code>foo#</code> (if there's no space before the <code>#</code>) as well, even without a token immediately after it. It sounds like doing that might be more consistent with the current behavior of <code>r#</code>.</p>



<a name="238531443"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238531443" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238531443">(May 12 2021 at 19:34)</a>:</h4>
<p>I'm also surprised that <code>r# foo</code> tokenizes as <code>r#foo</code>, FWIW.</p>



<a name="238531573"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238531573" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238531573">(May 12 2021 at 19:34)</a>:</h4>
<p>But if we have precedent from it, it seems fine to say that the migration is to <code>r #foo</code> and not <code>r# foo</code>, and to reserve more...</p>



<a name="238566034"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238566034" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238566034">(May 13 2021 at 00:53)</a>:</h4>
<p><span class="user-mention silent" data-user-id="125270">scottmcm</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238531443">said</a>:</p>
<blockquote>
<p>I'm also surprised that <code>r# foo</code> tokenizes as <code>r#foo</code>, FWIW.</p>
</blockquote>
<p>It's different, actually.</p>
<ul>
<li>The lexer tokenizes <code>r#foo</code> as a raw identifier <code>foo</code>, <em>successfully</em>.</li>
<li>The lexer <em>tries</em> to tokenize <code>r# foo</code> as a raw string, because it sees <code>r#</code> and the next character <code> </code> cannot be the start of an identifier. It <em>failed</em> with <code>found invalid character</code> because it expects either <code>#</code> and <code>"</code> for a raw string.</li>
</ul>



<a name="238566143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238566143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238566143">(May 13 2021 at 00:55)</a>:</h4>
<p><span class="user-mention silent" data-user-id="256342">bstrie</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238522904">said</a>:</p>
<blockquote>
<blockquote>
<p>However, the RFC specified we should accept foo# bar as three tokens.</p>
</blockquote>
<p>as the author of the RFC, any "accept as three tokens" language should be read as "for the purposes of this RFC we don't actually care what the behavior is other than that it <em>won't</em> be a reserved prefix and therefore won't parse as one token; do whatever is most natural, most consistent, and least disruptive". I wasn't aware that <code>r# foo</code> didn't already parse as three tokens, for example; if I had know then I wouldn't have presumptively used the "three tokens" language. restrict the scope of the RFC as much as is reasonable.</p>
</blockquote>
<p>OK I got it. Thanks for the explanation.</p>



<a name="238592221"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238592221" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238592221">(May 13 2021 at 07:52)</a>:</h4>
<blockquote>
<div class="codehilite"><pre><span></span><code>error: unknown prefix on identifier: a#
 --&gt; tokens.rs:7:7
  |
7 | demo!(a#foo);
  |       ^^ help: try using whitespace here: `a # foo`
  |
  = note: prefixed identifiers are reserved for future use
</code></pre></div>
<p>Note that this syntactic reservation is whitespace-sensitive: any whitespace to either side of the intervening <code>#</code> will allow this code to compile. This provides a simple migration path for anyone who would be impacted by this change; they would need only change their macro invocations from <code>foo!(bar#qux)</code> to any of <code>foo!(bar # qux)</code> , <code>foo!(bar# qux)</code> , or <code>foo!(bar #qux)</code> . It is possible to automate this mechanical migration via rustfix.</p>
</blockquote>
<p>another part of the RFC that's relevant to this discussion / would have to be changed if <code>bar# qux</code> was disallowed</p>



<a name="238737138"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238737138" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238737138">(May 14 2021 at 08:55)</a>:</h4>
<p>Did we settle these semantics? I think we didn't quite :)</p>



<a name="238746967"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238746967" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238746967">(May 14 2021 at 10:36)</a>:</h4>
<p>I thought we could either deny anything beginning with <code>foo#</code>or change the behavior of <code>r#</code>. According to the discussion, it seems that we prefer the former (denying anything beginning with <code>foo#</code>).</p>



<a name="238746987"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238746987" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238746987">(May 14 2021 at 10:36)</a>:</h4>
<p>So we have</p>
<ul>
<li>
<p><code>foo## bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>foo#! bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>foo##! bar</code><br>
     error: reserved prefixes</p>
</li>
<li>
<p><code>foo##" bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>foo##bar</code><br>
    error: reserved prefixes</p>
</li>
<li>
<p><code>r"foo"#bar</code><br>
    3 tokens <code>"foo"</code>, <code>#</code>, <code>bar</code></p>
</li>
</ul>



<a name="238747083"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238747083" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238747083">(May 14 2021 at 10:37)</a>:</h4>
<p>But what about <code>r#foo#bar</code>? It looks like both <code>r"foo"#bar</code>(3 tokens) and <code>foo#bar</code>(reserved prefixes). Does this one matter?</p>



<a name="238763454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238763454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238763454">(May 14 2021 at 13:08)</a>:</h4>
<p>This approach would then unify the <code>ident#foo</code> and <code>ident#123</code> cases described in the RFC and also disallow certain instances of attributes on expressions (without an extra space), e.g.</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#![feature(stmt_expr_attributes)]</span><span class="w"></span>

<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">   </span><span class="k">return</span><span class="cp">#[allow(unused)]</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">       </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">42</span><span class="p">;</span><span class="w"></span>
<span class="w">       </span><span class="p">()</span><span class="w"></span>
<span class="w">   </span><span class="p">};</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>right?</p>



<a name="238764520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238764520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238764520">(May 14 2021 at 13:16)</a>:</h4>
<p>Exactly. I never thought of this kind of code before.</p>



<a name="238766866"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238766866" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238766866">(May 14 2021 at 13:32)</a>:</h4>
<p>Similarly for <code>if#[attr] expr { expr }</code> and <code>match#[attr] expr { match_arms }</code>. None of these are stable without <code>stmt_expre_attributes</code> as-is, but they can probably be used by macros. At least something like</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="fm">macro_rules!</span><span class="w"> </span><span class="n">eat_expression</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="cp">$e</span>:<span class="nc">expr</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="n">eat_expression</span><span class="o">!</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">if</span><span class="cp">#[attr]</span><span class="w"> </span><span class="kc">true</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>works on stable. (Thus I would believe that inside of attribute-macros, you could use attributes on expressions as well. Haven’t tested or otherwise double-checked that.)</p>



<a name="238771594"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238771594" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238771594">(May 14 2021 at 14:02)</a>:</h4>
<p>I think <span class="user-mention" data-user-id="407862">@lrh2000</span> that sounds great. More lex errors. We'll let crate guide us a bit.</p>



<a name="238771679"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238771679" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238771679">(May 14 2021 at 14:03)</a>:</h4>
<p>And <span class="user-mention" data-user-id="280891">@Frank Steffahn</span> I would expect the migration to cover those cases in particular, right?</p>



<a name="238774590"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238774590" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238774590">(May 14 2021 at 14:22)</a>:</h4>
<p>I’m just pointing out more potential breakage. Of course automatic migrations are supposed to make all of this not really a problem at all.</p>
<p>Also, I personally feel like disallowing all <code>ident#</code> use cases is a significant change from the original RFC. I don’t really care about the choice here: reserving all <code>ident#</code> uses <em>seems fine and reasonable</em>, especially since automatic migrations and probably some nice error messages in the future will make the change non-problematic.</p>
<p>However, I feel like the <a href="https://blog.rust-lang.org/2021/05/11/edition-2021.html">blog post</a> becomes somewhat out-of-date/inaccurate with this change since it explicitly says, “we've decided to reserve syntax for prefixed identifiers and literals: <code>prefix#identifier</code>, <code>prefix"string"</code>, <code>prefix'c'</code>, and <code>prefix#123</code>, where prefix can be any identifier”.</p>
<p>Over in <a class="stream-topic" data-stream-id="268952" href="/#narrow/stream/268952-edition-2021/topic/News.20articles">#edition 2021 &gt; News articles</a> there’s some other websites some of which also picked up the mention of <code>prefix#identifier</code> and <code>prefix#123</code>. The change to <code>prefix#</code> is then – in my view – a trade-off between communicating this change and what the actual benefits are. I’m not sure that I know of any particular benefit of reserving <code>prefix#</code> instead of just <code>prefix#ident</code> and <code>prefix#123</code> (but maybe I’m missing something?).</p>



<a name="238776694"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238776694" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238776694">(May 14 2021 at 14:35)</a>:</h4>
<blockquote>
<p>I’m not sure that I know of any particular benefit of reserving <code>prefix#</code> instead of just <code>prefix#ident</code> and <code>prefix#123</code> (but maybe I’m missing something?).</p>
</blockquote>
<p>So do you mean that we should change the behavior of <code>r#</code>? Otherwise, how do we deal with <code>fr#</code> (which is nothing more than a reserved prefix now but in future it may behave like <code>r#</code>)?</p>



<a name="238777193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238777193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238777193">(May 14 2021 at 14:38)</a>:</h4>
<p>(Of course, we can defer the issue and in the future we can always make <code>fr#</code> behave differently from <code>r#</code>, which is a little awkward at least in my opinion.)</p>



<a name="238778011"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238778011" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238778011">(May 14 2021 at 14:44)</a>:</h4>
<p>(And we have to talk about corner cases like <code>foo##"bar"</code>, <code>foo##!bar</code> again, which complicates the lexer rules and seems to be unnecessary.)</p>



<a name="238778422"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238778422" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238778422">(May 14 2021 at 14:47)</a>:</h4>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238776694">said</a>:</p>
<blockquote>
<p>So do you mean that we should change the behavior of <code>r#</code>?</p>
</blockquote>
<p>Well, yes and no. I’m saying that reserving <code>prefix#</code> is reasonable but needs to be communicated and may have little benefix. I’m also – indeed – of the opinion that <strong>if</strong> we <em>don’t</em> disallow <code>prefix#</code> in general <strong>then</strong>, yes, we should probably allow <code>r#</code>. So in a sense change the behavior of <code>r#</code>. Although the only “change” of course is that code that didn’t tokenize before tokenizes now, and could possibly be used in macros.</p>
<p>I also have used <code>#</code> in macros before, although IIRC only at the beginning of the macro (so none of the cases discussed here would apply), especially in <code>macro_rules</code> macros to make recursive calls that don’t clash with the intended use case. I liked the symmetric situation of “you can break up <code>foo#bar</code> in any way you want to make it work in the new edition” in the RFC, so I guess I have a very slight preference to not disallowing <code>prefix#</code>. I haven’t thought too hard about all potential corner cases in lexing so if it turns out that allowing <code>prefix#</code> seems to be unnecessarily hard and complicated then that might be a good reason not to take that approach.</p>



<a name="238780071"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238780071" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238780071">(May 14 2021 at 14:58)</a>:</h4>
<p>Roughly I think our views are actually the same.</p>
<blockquote>
<p>I haven’t thought too hard about all potential corner cases in lexing so if it turns out that allowing prefix# seems to be unnecessarily hard and complicated then that might be a good reason not to take that approach.</p>
</blockquote>
<p>If we allow <code>foo# bar</code>, there is no reasons we don't allow <code>foo## bar</code>, <code>foo##! bar</code> and so on. But we cannot allow <code>foo##"</code> since we use <code>r##"</code> to denote a raw string. It means that we may have to look forward at a lot of <code>#</code> before making the final decision. I'm not sure whether it is <em>unnecessarily hard and complicated</em> enough. At least, we can avoid this by disallowing <code>foo#</code>.</p>



<a name="238781139"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238781139" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238781139">(May 14 2021 at 15:05)</a>:</h4>
<p>I don't feel that making <code>foo#</code> an error is so very different from <code>foo#bar</code> being reserved</p>



<a name="238798336"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238798336" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238798336">(May 14 2021 at 17:07)</a>:</h4>
<p>Okay, I guess the <code>r##"string"##</code> example <em>is</em> a reasonable argument <em>against</em> allowing <code>foo#</code> in <em>every</em> case other than <code>foo#ident</code> and <code>foo#123</code>. I suppose there’s also other options like for example</p>
<ul>
<li>disallowing <code>foo#</code> unless it’s directly followed by whitespace (in which case it becomes 2 tokens), or</li>
<li>disallowing <code>foo#</code> unless it’s directly followed by whitespace or <code>[</code> or <code>!</code>, which would still allow the attributes examples</li>
</ul>



<a name="238946298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238946298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238946298">(May 16 2021 at 03:43)</a>:</h4>
<p>So we do have several options:</p>
<ul>
<li>disallowing anything beginning with <code>foo#</code></li>
<li>disallowing <code>foo#</code> unless it’s directly followed by whitespace, <em>and</em> adjust the behavior of <code>r#</code></li>
<li>disallowing <code>foo#</code> unless it’s directly followed by whitespace or <code>[</code> or <code>!</code>, <em>and</em> adjust the behavior of <code>r#</code></li>
</ul>
<p>I guess none of the above is significantly better than others and none of the above is significantly worse than others, which makes the selection difficult. So maybe we can just pick the simplest one (i.e. the first one).</p>
<p>I've opened a PR <a href="https://github.com/rust-lang/rust/issues/85359">#85359</a> for this. Of course, we can still adjust its behavior during the review of that PR.</p>



<a name="238979476"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238979476" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238979476">(May 16 2021 at 13:39)</a>:</h4>
<p>Just noticed that the RFC and your PR also disallow things like</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">match</span><span class="s">"foo"</span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">_</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">()</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>As long as that’s something we actually want to forbid, I don’t see any problem in disallowing <code>return</code>/<code>match</code>/<code>if</code>/… followed by attributes without a space either. In case we <em>do</em> want to e.g. still allow existing keywords followed by <code>#</code>, that would solve the problem with attributes as well. The RFC does forbid <code>IDENTIFIER_OR_KEYWORD</code> prefixes but it doesn’t really give any reason why non-keywords wouldn’t be enough. So this could give rise to some more reasonable alternatives</p>
<ul>
<li>disallow anything starting with <code>foo#</code>, <code>foo'</code>, or <code>foo"</code>  unless <code>foo</code> is a keyword</li>
<li>disallow anything starting with <code>foo#</code>, <code>foo'</code>, or <code>foo"</code> unless <code>foo</code> is one of the keywords <code>break</code>, <code>if</code>, <code>in</code>, <code>match</code>, <code>return</code>, <code>while</code>, <code>continue</code>. (Further keywords such as <code>box</code>, or <code>yield</code> could be added later without any problems, once they are stabilized.)</li>
</ul>
<p>If <code>foo</code> is a keyword / one of the allowed keywords, then <code>foo#……</code> should parse the same as <code>foo #……</code> with a space after the keyword</p>
<p>AFAICT (correct me if I’m wrong) the keywords listed above should be all the cases where currently an IDENTIFIER_OR_KEYWORD can be followed by a string/char literal or an attribute or a lifetime/label.</p>
<p>Either of these approaches should have the effect that it doesn’t break (i.e. introduce the need for changes to) any working code that uses string or char literals without preceding spaces or any code that uses procedural attribute macros (which are able to parse/accept attributes on expressions).</p>



<a name="238979787"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238979787" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238979787">(May 16 2021 at 13:43)</a>:</h4>
<p>Actually, your PR also disallows</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="o">'</span><span class="na">label</span>: <span class="nc">loop</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">continue</span><span class="nl">'label</span><span class="p">;</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>With either of the proposals above, this would be fixed, too (I’ve added <code>continue</code> to the list of keywords).</p>



<a name="238982155"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238982155" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238982155">(May 16 2021 at 14:24)</a>:</h4>
<p>The impact of the PR on lifetimes is relevant in ordinary macros, too, where you can currently do</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="fm">macro_rules!</span><span class="w"> </span><span class="n">qux</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="cp">$i</span>:<span class="nc">ident</span><span class="w"> </span><span class="cp">$l</span>:<span class="nc">lifetime</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">qux</span><span class="o">!</span><span class="p">(</span><span class="n">ident</span><span class="o">'</span><span class="na">lifetime</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>So maybe allowing <code>ident'lifetime</code> in general is preferable?</p>



<a name="238988685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238988685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238988685">(May 16 2021 at 16:03)</a>:</h4>
<p>Yeah, thanks a lot for your feedback. I think your argument makes sense and I should consider this situations more carefully.</p>



<a name="238994267"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238994267" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238994267">(May 16 2021 at 17:36)</a>:</h4>
<blockquote>
<p>AFAICT (correct me if I’m wrong) the keywords listed above should be all the cases where currently an IDENTIFIER_OR_KEYWORD can be followed by a string/char literal or an attribute or a lifetime/label.</p>
</blockquote>
<p>Actually, <code>where</code> can be followed by lifetimes, too.</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">fn</span> <span class="nf">foo</span><span class="o">&lt;'</span><span class="na">a</span><span class="p">,</span><span class="w"> </span><span class="o">'</span><span class="na">b</span><span class="o">&gt;</span><span class="p">()</span><span class="w"> </span><span class="k">where</span><span class="o">'</span><span class="na">a</span>:<span class="o">'</span><span class="na">b</span> <span class="p">{}</span><span class="w"></span>
</code></pre></div>



<a name="238994898"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238994898" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238994898">(May 16 2021 at 17:47)</a>:</h4>
<p>And <code>let</code> can be followed by char and string literals</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">if</span><span class="w"> </span><span class="kd">let</span><span class="s">""</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">""</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
</code></pre></div>



<a name="238994997"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238994997" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238994997">(May 16 2021 at 17:49)</a>:</h4>
<p>I wonder if there is a rust minifier around somewhere</p>



<a name="238995049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238995049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238995049">(May 16 2021 at 17:50)</a>:</h4>
<p>or code golf, I guess</p>



<a name="238995133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238995133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238995133">(May 16 2021 at 17:51)</a>:</h4>
<p>Going back to attribute macros or <code>$e:expr</code> macro arguments: <code>for</code> can be followed by string or char literal (patterns) as well</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[proc_macro_attribute]</span><span class="w"></span>
<span class="k">pub</span><span class="w"> </span><span class="k">fn</span> <span class="nf">noop</span><span class="p">(</span><span class="n">_</span>: <span class="nc">TokenStream</span><span class="p">,</span><span class="w"> </span><span class="n">_</span>: <span class="nc">TokenStream</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nc">TokenStream</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="o">&lt;</span><span class="n">_</span><span class="o">&gt;</span>::<span class="n">default</span><span class="p">()</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[noop]</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">foo</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">for</span><span class="s">""</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="fm">macro_rules!</span><span class="w"> </span><span class="n">ignore_expression</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="cp">$e</span>:<span class="nc">expr</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="n">ignore_expression</span><span class="o">!</span><span class="p">(</span><span class="k">for</span><span class="s">""</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="p">()</span><span class="w"> </span><span class="p">{});</span><span class="w"></span>
</code></pre></div>



<a name="238995344"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238995344" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank Steffahn <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238995344">(May 16 2021 at 17:55)</a>:</h4>
<p>So the list is now <code>break</code>, <code>if</code>, <code>in</code>, <code>match</code>, <code>return</code>, <code>while</code>, <code>continue</code>, <code>where</code>, <code>for</code>.</p>



<a name="238998356"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/238998356" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#238998356">(May 16 2021 at 18:49)</a>:</h4>
<p><code>for</code> is notable since it ends in <code>r</code>; apparently <code>for""</code> does not tokenize as a raw string literal</p>



<a name="239291307"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/239291307" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#239291307">(May 18 2021 at 17:15)</a>:</h4>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238746967">said</a>:</p>
<blockquote>
<p>I thought we could either deny anything beginning with <code>foo#</code>or change the behavior of <code>r#</code>. According to the discussion, it seems that we prefer the former (denying anything beginning with <code>foo#</code>).</p>
</blockquote>
<p>We discussed this in the <span class="user-group-mention" data-user-group-id="1977">@T-lang</span> meeting today, and the consensus was that being consistent with the current behaviour of <code>r#</code> sounds good.</p>
<p>(Especially since we can always make it allowed later, so this is the easy consensus.)</p>



<a name="240097771"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097771" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097771">(May 24 2021 at 19:13)</a>:</h4>
<p>Hey folks</p>



<a name="240097798"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097798" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097798">(May 24 2021 at 19:13)</a>:</h4>
<p>I'm somewhat convinced by <span class="user-mention" data-user-id="123856">@Vadim Petrochenkov</span>'s assertion that we should make this change in the Rust parser area</p>



<a name="240097865"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097865" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097865">(May 24 2021 at 19:14)</a>:</h4>
<p>I'm curious <span class="user-mention" data-user-id="407862">@lrh2000</span> how much you've explored this and what it would mean</p>



<a name="240097878"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097878" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097878">(May 24 2021 at 19:14)</a>:</h4>
<p>I have to admit I'm getting worried about shipping this change for the edition</p>



<a name="240097882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097882">(May 24 2021 at 19:14)</a>:</h4>
<p>(This may want to be in #t-lang)</p>



<a name="240097922"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097922" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097922">(May 24 2021 at 19:14)</a>:</h4>
<p>There is also the option of shipping a scaled back <code>k#foo</code>, but it seemed like we felt that could be done across all editions</p>



<a name="240097940"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097940" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097940">(May 24 2021 at 19:15)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240097882">said</a>:</p>
<blockquote>
<p>(This may want to be in #t-lang)</p>
</blockquote>
<p>It may, but this is where we've been conversing</p>



<a name="240097959"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240097959" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240097959">(May 24 2021 at 19:15)</a>:</h4>
<p>so I figured most of the folks involved would see it here</p>



<a name="240181211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240181211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240181211">(May 25 2021 at 12:46)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116009">nikomatsakis</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240097865">said</a>:</p>
<blockquote>
<p>I'm curious <span class="user-mention silent" data-user-id="407862">lrh2000</span> how much you've explored this and what it would mean</p>
</blockquote>
<p>So I'd like to first ask whether we have confirmed that we want to see <code>match"foo"</code> to be compiled successfully. I'm not sure because it seems that nobody expressed strong views on this problem clearly after the original suggestion made by <span class="user-mention" data-user-id="280891">@Frank Steffahn</span> .</p>
<p>If we do, we should make this change in parser. The lexer is not aware of the keyword list.</p>
<ul>
<li>
<p>Whenever <code>rustc_parse</code> sees an ident in <code>parse_ident</code>, it can check whether or not a <code>#</code> or a string literal is followed immediately (without any whitespaces in the middle). If it is, it emits an error because it has found a reserved prefix.</p>
</li>
<li>
<p>If <code>rustc_parse</code> sees a keyword, it shouldn't go through the <code>parse_ident</code> path (which is the normal case, otherwise, if the <code>parse_ident</code> is still called, we'll emit an error <code>expected identifier, found keyword</code>). Since we know we will never use a keyword as a prefix, we can safely accept the code.</p>
</li>
</ul>
<p>But what I am concerned is that we have another parser that lives in <code>rustc_expand</code> that designs for the macro definitions and invocations. If we make this change only in parser, we can successfully convert <code>print#once</code> into a <code>TokenStream</code> with three tokens <code>print</code>, <code>#</code> and <code>once</code>. So consider the following code segment</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="fm">macro_rules!</span><span class="w"> </span><span class="n">foo</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="n">print</span><span class="w"> </span>#<span class="w"> </span><span class="n">once</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="p">};</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">foo</span><span class="o">!</span><span class="p">(</span><span class="n">print</span>#<span class="n">once</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>Since we can successfully convert <code>print#once</code> into a <code>TokenStream</code>, <code>rustc_expand</code> will get this <code>TokenStream</code> at the macro invocation <code>foo!(print#once)</code>. But it just matches the first macro rule which is also a <code>TokenStream</code> with three tokens <code>print</code>, <code>#</code> and <code>once</code>. The whole matching is not delegated to <code>rustc_parse</code> since there are no meta var decls (see <a href="https://github.com/rust-lang/rust/blob/cdbe2888979bb8797b05f0d58a6f6e60753983d2/compiler/rustc_expand/src/mbe/macro_parser.rs#L597-L601">https://github.com/rust-lang/rust/blob/cdbe2888979bb8797b05f0d58a6f6e60753983d2/compiler/rustc_expand/src/mbe/macro_parser.rs#L597-L601</a>).</p>
<p>So we have to check reserved prefixes here for every token, in case we accidentally accept the reserved prefix. I don't think it is a good idea. <strong>But if we decide to do so, how about just checking reserved prefixes while we are generating any <code>TokenStream</code>s?  It lives in <code>rustc_parse/src/lexer</code>, a position between the lexer and the parser.</strong> Does it sound good?</p>
<p>cc <span class="user-mention" data-user-id="123856">@Vadim Petrochenkov</span></p>



<a name="240193022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240193022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240193022">(May 25 2021 at 14:10)</a>:</h4>
<p><code>rustc_parse/src/lexer</code> is place where lexer-style tokens are converted to parser-style tokens.<br>
If you want to do something with all tokens in a centralized way, then it's a good place to do that.</p>



<a name="240193189"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240193189" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240193189">(May 25 2021 at 14:11)</a>:</h4>
<p>It's a part of the parser logically, it has access to <code>rustc_span</code>, knows about keywords, etc.</p>



<a name="240197129"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240197129" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240197129">(May 25 2021 at 14:37)</a>:</h4>
<p>I would not expect <code>match"foo"</code> to compile</p>



<a name="240197248"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240197248" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240197248">(May 25 2021 at 14:38)</a>:</h4>
<p>I am however increasingly convinced that we should move this logic to the parser</p>



<a name="240201756"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240201756" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240201756">(May 25 2021 at 15:07)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116009">nikomatsakis</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240197248">said</a>:</p>
<blockquote>
<p>I am however increasingly convinced that we should move this logic to the parser</p>
</blockquote>
<p>Why? I thought making <code>match"foo"</code> to compile is the most significant benefit by moving this logic to the parser. Could you please explain more for this?</p>



<a name="240202015"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240202015" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240202015">(May 25 2021 at 15:08)</a>:</h4>
<blockquote>
<p>If you want to do something with all tokens in a centralized way, then it's a good place to do that.</p>
</blockquote>
<p>Yeah, I think it is the case.</p>



<a name="240214327"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240214327" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240214327">(May 25 2021 at 16:29)</a>:</h4>
<p>I guess I don't have a strong opinion about whether <code>match"foo"</code> compiles</p>



<a name="240214335"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240214335" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240214335">(May 25 2021 at 16:29)</a>:</h4>
<p>but the benefit I see is that the lexer remains edition agnostic</p>



<a name="240214389"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240214389" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240214389">(May 25 2021 at 16:30)</a>:</h4>
<p>when we designed the proc macro API, we tried to design it to give back simple tokens with "adjacency information" so that we could add new operators like <code>&lt;&lt;&lt;&lt;</code> in the future without breaking back-compat</p>



<a name="240214454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240214454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240214454">(May 25 2021 at 16:30)</a>:</h4>
<p>I see the appeal of using that to parse <code>k#foo</code> differently from <code>foo</code> etc</p>



<a name="240214470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240214470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240214470">(May 25 2021 at 16:30)</a>:</h4>
<p>while still tokenizing it as we ever did</p>



<a name="240214503"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240214503" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240214503">(May 25 2021 at 16:30)</a>:</h4>
<p>or, more to the point, <code>f"foo"</code></p>



<a name="240215206"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240215206" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240215206">(May 25 2021 at 16:35)</a>:</h4>
<p>note that there are already things that enforce mandatory surrounding whitespace; <code>let x = .. ..;</code> compiles, but <code>let x = ....;</code> does not.</p>



<a name="240220164"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240220164" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240220164">(May 25 2021 at 17:12)</a>:</h4>
<p>(aside, but I thought of one more thing that reserved prefixes might be useful for: in macros there's an open question of which scope identifiers ought to be resolved in, and this might be disambiguated by having <code>call#foo</code> or <code>def#foo</code>)</p>



<a name="240224083"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240224083" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240224083">(May 25 2021 at 17:41)</a>:</h4>
<p>I don't think I'd want to see <code>#</code> used for that. We already have the <code>::</code> operator.</p>



<a name="240224136"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240224136" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240224136">(May 25 2021 at 17:41)</a>:</h4>
<p>If there's some way we want to change name resolution, I think that should be a <code>newthing::some::name</code>, where <code>newthing</code> can be <code>k#newthing</code> if need be.</p>



<a name="240224798"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240224798" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240224798">(May 25 2021 at 17:47)</a>:</h4>
<p><span class="user-mention" data-user-id="407862">@lrh2000</span> <span class="user-mention" data-user-id="123856">@Vadim Petrochenkov</span> If we check reserved prefixes in <code>rustc_parse/src/lexer</code>, does that allow proc macros to parse and give meaning to reserved prefixes, or will that catch reserved prefixes before macros can process them?</p>



<a name="240224967"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240224967" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240224967">(May 25 2021 at 17:48)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240224083">said</a>:</p>
<blockquote>
<p>I don't think I'd want to see <code>#</code> used for that. We already have the <code>::</code> operator.</p>
</blockquote>
<p>would that work for defining types that get injected into the caller's scope? I was thinking of <code>struct call#Foo</code></p>



<a name="240225034"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240225034" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240225034">(May 25 2021 at 17:48)</a>:</h4>
<p><span class="user-mention" data-user-id="256342">@bstrie</span> As in a macro defining types, and wanting to explicitly ignore hygiene?</p>



<a name="240225091"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240225091" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bstrie <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240225091">(May 25 2021 at 17:49)</a>:</h4>
<p>yes, this is the open question that rustc internally uses <code>#[rustc_macro_transparency]</code> to work around</p>



<a name="240225125"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240225125" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240225125">(May 25 2021 at 17:49)</a>:</h4>
<p>That seems like a matter of macro syntax, and I think I'd expect that to look more like <code>caller_scope!(Foo)</code> rather than something involving <code>#</code>.</p>



<a name="240225326"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240225326" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240225326">(May 25 2021 at 17:50)</a>:</h4>
<p>Also, in many cases a macro may want to combine that with some kind of gensym mechanism ("give me a fresh identifier that can't conflict with anything").</p>



<a name="240226132"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240226132" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240226132">(May 25 2021 at 17:56)</a>:</h4>
<p>Regarding lexer vs parser, two things I'd like to make sure we address:</p>



<a name="240226133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240226133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240226133">(May 25 2021 at 17:56)</a>:</h4>
<p>1) I want to make sure proc macros can't assign meaning to reserved prefixes that would conflict with the language assigning meaning to them in the future. I don't want to end up in a situation where people start using them and we either have to say "hey, we said this would break in the future and we're breaking it" but still cause problems for people, or we have to work around such macros anyway and lose the intended benefit of reserved prefixes.</p>



<a name="240226319"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240226319" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240226319">(May 25 2021 at 17:58)</a>:</h4>
<p>2) I'd like to make sure reserved syntax can itself be used in the definitions of macros themselves, and similarly "early" things. That may be less of an issue if only proc macros can "see" the reserved syntax, but I still want to verify that whatever solution we go with doesn't limit our ability to use reserved syntax in the definition of arbitrary language constructs, including macros.</p>



<a name="240227396"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240227396" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240227396">(May 25 2021 at 18:05)</a>:</h4>
<p>If we can accomplish both of those things while still having an edition-independent lexer, I'm all for it. For clarity, I don't have any preference whether this happens in the lexer or the parser; I just care about both of the above considerations.</p>



<a name="240233394"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240233394" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240233394">(May 25 2021 at 18:52)</a>:</h4>
<p>There was an idea someone (I forget by whom, sorry) brought up a while ago that seemed like it could help for <code>macro_rules</code> at least: make "unknown tokens" (whatever that means) match things like <code>expr</code> and <code>ident</code> etc.</p>
<p>That way a macro written <code>{ ( k#foo ) =&gt; { ... }, ( $i:ident ) =&gt; { ... } }</code> can still work, but anything written <code>{ ( $i:ident ) =&gt; { ... }, ( k#foo ) =&gt; { ... } }</code> would just never match the second one, as it'd get picked up by the first arm and then give an error that it's not a valid ident.</p>
<p>That wouldn't help proc macros, but they're always a bit of a wild west...</p>



<a name="240239815"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240239815" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Daniel Henry-Mantilla <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240239815">(May 25 2021 at 19:44)</a>:</h4>
<p><span class="user-mention silent" data-user-id="125270">scottmcm</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240233394">said</a>:</p>
<blockquote>
<p>There was an idea someone (I forget by whom, sorry) brought up a while ago that seemed like it could help for <code>macro_rules</code> at least: make "unknown tokens" (whatever that means) match things like <code>expr</code> and <code>ident</code> etc.</p>
<p>That way a macro written <code>{ ( k#foo ) =&gt; { ... }, ( $i:ident ) =&gt; { ... } }</code> can still work, but anything written <code>{ ( $i:ident ) =&gt; { ... }, ( k#foo ) =&gt; { ... } }</code> would just never match the second one, as it'd get picked up by the first arm and then give an error that it's not a valid ident.</p>
<p>That wouldn't help proc macros, but they're always a bit of a wild west...</p>
</blockquote>
<p>I believe it was me <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span> To further clarify: a <code>k#foo</code> that has not yet been given a parseable meaning in the compiler would thus:</p>
<ul>
<li>start by being "valid components" of any macro capture kind;</li>
<li>but then end with a parsing error due to <code>k#foo</code> having no meaning and thus there not being a valid parsing for that capture kind.</li>
</ul>
<p>An example of such behavior can currently be observed with ambiguous prefixes:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="fm">macro_rules!</span><span class="w"> </span><span class="n">m</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="cp">$_</span>:<span class="nc">expr</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">();</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="cp">$_</span>:<span class="nc">lifetime</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">();</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
<span class="n">m</span><span class="o">!</span><span class="p">(</span><span class="w"> </span><span class="o">'</span><span class="na">a</span><span class="w"> </span><span class="p">);</span><span class="w"> </span><span class="c1">// Error, `'a` is not a valid expr.</span>
</code></pre></div>
<ul>
<li><a href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2018&amp;gist=001d98f6f9fc87dd91a028147424ff37">Playground</a></li>
</ul>
<p>By doing so, indeed, we'd future-proof against <code>macro_rules!</code> captures as much as possible, since the only way one would match against things such as <code>k#foo</code> would then be either literally, or through <code>$($_:tt)*</code> kind of captures.</p>
<p>In other words, or as a recap, since any potential keyword <em>could</em> become a valid component for whatever capture we are dealing with,</p>
<ul>
<li>we cannot reject macro arms based on captures.</li>
</ul>
<p>But since they currently have no meaning assigned / they <em>could</em> become an invalid component for those very specific captures,</p>
<ul>
<li>we cannot let the capture be fully parsed.</li>
</ul>



<a name="240267390"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240267390" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240267390">(May 26 2021 at 00:38)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240224798">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> <span class="user-mention silent" data-user-id="123856">Vadim Petrochenkov</span> If we check reserved prefixes in <code>rustc_parse/src/lexer</code>, does that allow proc macros to parse and give meaning to reserved prefixes, or will that catch reserved prefixes before macros can process them?</p>
</blockquote>
<p>It still leads to edition-dependent <code>TokenStream</code>, so the case should be the same as we check reserved prefixes in <code>rustc_lexer</code>. The benefit is just we have available keyword list. But as <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240214335">said</a> by <span class="user-mention" data-user-id="116009">@nikomatsakis</span>:</p>
<blockquote>
<p>but the benefit I see is that the lexer remains edition agnostic</p>
</blockquote>
<p>If  instead, what we really want is an edition-agnostic lexer (I think here the lexer means <code>TokenStream</code>), we have to defer the check again, maybe in a position where we actually extracts tokens from any <code>TokenStream</code> to parse (possibly <code>Parser::bump()</code> or something similar?).</p>



<a name="240268074"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240268074" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240268074">(May 26 2021 at 00:52)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240226319">said</a>:</p>
<blockquote>
<p>2) I'd like to make sure reserved syntax can itself be used in the definitions of macros themselves, and similarly "early" things. That may be less of an issue if only proc macros can "see" the reserved syntax, but I still want to verify that whatever solution we go with doesn't limit our ability to use reserved syntax in the definition of arbitrary language constructs, including macros.</p>
</blockquote>
<p>Do you mean that we can safely write the below code segment?</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="fm">macro_rules!</span><span class="w"> </span><span class="n">foo</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="p">(</span><span class="n">print</span>#<span class="n">once</span><span class="p">)</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="p">};</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">foo</span><span class="o">!</span><span class="p">(</span><span class="n">print</span><span class="w"> </span>#<span class="w"> </span><span class="n">once</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>If we can, we have to explicitly distinguish macro invocations from macro definitions, since we'll check reserved prefixes for invocations but not for definitions. It complicates the problem and I don't understand why we should do this, could you please explain a bit more?</p>



<a name="240278179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240278179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240278179">(May 26 2021 at 04:20)</a>:</h4>
<p><span class="user-mention" data-user-id="407862">@lrh2000</span> That's not what I mean, sorry.</p>



<a name="240278260"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240278260" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240278260">(May 26 2021 at 04:23)</a>:</h4>
<p><span class="user-mention" data-user-id="407862">@lrh2000</span> I mean that reserved syntax should, itself, be potentially usable as part of the definition of a macro. For instance, we could decide that <code>k#xyz_macro</code> is the new keyword for defining a new kind of macro, and then <code>k#xyz_macro! foo { ... }</code> should work. Or, <code>m#gensym</code> could be part of a new syntax for defining a new identifier inside a macro.</p>



<a name="240300192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240300192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240300192">(May 26 2021 at 09:49)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240278260">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> I mean that reserved syntax should, itself, be potentially usable as part of the definition of a macro. For instance, we could decide that <code>k#xyz_macro</code> is the new keyword for defining a new kind of macro, and then <code>k#xyz_macro! foo { ... }</code> should work. Or, <code>m#gensym</code> could be part of a new syntax for defining a new identifier inside a macro.</p>
</blockquote>
<p>It should be straightforward in lexer. We can manage to do this in parser, i.e. we are of course <em>able</em> to glue <code>k</code>, <code>#</code> and <code>xyz_macro</code> together, as suggested <a href="https://github.com/rust-lang/rust/issues/84979#issuecomment-843929357">here</a>, although personally I think it is less elegant.</p>



<a name="240300672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240300672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240300672">(May 26 2021 at 09:54)</a>:</h4>
<blockquote>
<p>1) I want to make sure proc macros can't assign meaning to reserved prefixes that would conflict with the language assigning meaning to them in the future. I don't want to end up in a situation where people start using them and we either have to say "hey, we said this would break in the future and we're breaking it" but still cause problems for people, or we have to work around such macros anyway and lose the intended benefit of reserved prefixes.</p>
</blockquote>
<p>It is also straightforward in lexer. If we manage to do this in parser, we'd better to find a centralized way, <code>Parser::bump()</code> for example, as I've already mentioned before. Personally I think the latter is less elegant, too.</p>



<a name="240471431"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240471431" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240471431">(May 27 2021 at 13:59)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/240226133">said</a>:</p>
<blockquote>
<p>1) I want to make sure proc macros can't assign meaning to reserved prefixes that would conflict with the language assigning meaning to them in the future. I don't want to end up in a situation where people start using them and we either have to say "hey, we said this would break in the future and we're breaking it" but still cause problems for people, or we have to work around such macros anyway and lose the intended benefit of reserved prefixes.</p>
</blockquote>
<p>I'm not sure what I think about this goal. I think there is a real tension about "enable exploration" here. I actually think I might see it as a <em>benefit</em> that people can write procedural macros to test out things which may eventually become part of the standard language.</p>



<a name="240471906"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240471906" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240471906">(May 27 2021 at 14:02)</a>:</h4>
<p>It's probably worth distinguishing <em>decorators</em> from other sorts of procedural macros</p>



<a name="240471920"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240471920" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240471920">(May 27 2021 at 14:02)</a>:</h4>
<p>We're back in a kind of circular point</p>



<a name="240471941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240471941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240471941">(May 27 2021 at 14:02)</a>:</h4>
<p>I am getting very worried about schedule here; this doesn't seem to be coming together</p>



<a name="240471965"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240471965" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240471965">(May 27 2021 at 14:03)</a>:</h4>
<p>I'm going to try and write up a doc to summaize the paths forward</p>



<a name="240497140"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240497140" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240497140">(May 27 2021 at 16:46)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> There's a whole world of unreserved syntax for macros to use. To be, it seems like the point of reservation is to clear a space for future use by the language so that we stop having to worry about whether we might happen to break a macro. Then we can systematically use that reserved syntax for just about anything we might add outside of an edition boundary.</p>



<a name="240497199"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240497199" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240497199">(May 27 2021 at 16:47)</a>:</h4>
<p>We also have the ability to unreserve part of that space at any time if it represents syntax we want to enable experimentation with.</p>



<a name="240497247"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240497247" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240497247">(May 27 2021 at 16:47)</a>:</h4>
<p>In this case, I think we're trying to make the language more maintainable.</p>



<a name="240497399"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240497399" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240497399">(May 27 2021 at 16:48)</a>:</h4>
<p>As well as making maintenance scale; a full crater run may not be an option forever.</p>



<a name="240534249"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534249" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534249">(May 27 2021 at 21:26)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> we would not break macros</p>



<a name="240534255"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534255" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534255">(May 27 2021 at 21:26)</a>:</h4>
<p>they would keep working</p>



<a name="240534283"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534283" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534283">(May 27 2021 at 21:27)</a>:</h4>
<p>(if we took the approach of manging this stuff in the parser)</p>



<a name="240534298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534298">(May 27 2021 at 21:27)</a>:</h4>
<p>at least, procedural macros would</p>



<a name="240534310"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534310" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534310">(May 27 2021 at 21:27)</a>:</h4>
<p>macro-rules is more problematic, because of the fact that they can invoke the parser</p>



<a name="240534413"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534413" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534413">(May 27 2021 at 21:28)</a>:</h4>
<p>(which is where <span class="user-mention" data-user-id="232018">@Daniel Henry-Mantilla</span>'s proposal comes in)</p>



<a name="240534505"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534505" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534505">(May 27 2021 at 21:29)</a>:</h4>
<p>I'm not sure if that works, and I'm grumpy that the future proofing we tried to do on macro fragments didn't work, but oh well :)</p>



<a name="240534590"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534590" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534590">(May 27 2021 at 21:30)</a>:</h4>
<p>...though...we could make the <em>parser</em> hard error</p>



<a name="240534630"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/240534630" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#240534630">(May 27 2021 at 21:30)</a>:</h4>
<p>(i.e., not "abort parsing")</p>



<a name="241080093"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241080093" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241080093">(Jun 01 2021 at 21:47)</a>:</h4>
<p>OK I wrote up some notes on reserved prefies <a href="https://hackmd.io/YLe7viGLTu2PfE5sQO4v0w">https://hackmd.io/YLe7viGLTu2PfE5sQO4v0w</a> -- these are also based on discussions with <span class="user-mention" data-user-id="310399">@Mara</span> in the meeting today.</p>



<a name="241080105"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241080105" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241080105">(Jun 01 2021 at 21:47)</a>:</h4>
<p>Our conclusion was that we should stay the course with lexer errors</p>



<a name="241080117"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241080117" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241080117">(Jun 01 2021 at 21:47)</a>:</h4>
<p>(And make <code>match"foo"</code> a lexer error too)</p>



<a name="241080689"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241080689" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241080689">(Jun 01 2021 at 21:54)</a>:</h4>
<p><span class="user-mention" data-user-id="407862">@lrh2000</span> can you update as to the current status of the implementation? I'm feeling a bit lost :)</p>



<a name="241147610"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241147610" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241147610">(Jun 02 2021 at 13:23)</a>:</h4>
<blockquote>
<p><span class="user-mention silent" data-user-id="407862">lrh2000</span> can you update as to the current status of the implementation? I'm feeling a bit lost :)</p>
</blockquote>
<p>Since I have originally implemented it as a lexer error, I don't think I get much more things to do now, especially if we also decide to forbid stuffs like <code>match#foo</code>.</p>
<p>There is a PR <a href="https://github.com/rust-lang/rust/issues/85359">#85359</a>, and now I think what I still need to do is to fix the issue about lifetimes (if we want to allow <code>continue'label</code>). Is there anything else?</p>



<a name="241148191"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241148191" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241148191">(Jun 02 2021 at 13:27)</a>:</h4>
<p>Let me take a look</p>



<a name="241148260"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241148260" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241148260">(Jun 02 2021 at 13:28)</a>:</h4>
<p>I'm not sure about lifetimes, that wasn't covered in the RFC I guess</p>



<a name="241151830"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241151830" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241151830">(Jun 02 2021 at 13:53)</a>:</h4>
<p>Yes.</p>
<p>Now the simple implementation does not look at characters after <code>continue'</code> when emitting the reserved prefix error, so it disallows <code>continue'label</code>. If we want to allow it, I think we have to examine those characters ahead (but not eat them) in the lexer.</p>



<a name="241152103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/241152103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#241152103">(Jun 02 2021 at 13:55)</a>:</h4>
<p>And just like <code>match"foo"</code>, it won't cause any real problems because it can always be fixed by <code>continue 'label</code>.</p>



<a name="242614198"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242614198" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242614198">(Jun 14 2021 at 15:24)</a>:</h4>
<p><span class="user-mention" data-user-id="407862">@lrh2000</span> I left some review comments -- do you think you'll have time to address the changes? I or <span class="user-mention" data-user-id="310399">@Mara</span> can also push commits to your branch if you're busy</p>



<a name="242615701"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242615701" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242615701">(Jun 14 2021 at 15:34)</a>:</h4>
<p>Sorry I am indeed unexpectedly busy in June. (I'll be free by early July, and before that I can spend fairly limited time on this.)</p>



<a name="242615755"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242615755" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242615755">(Jun 14 2021 at 15:34)</a>:</h4>
<p>But I'd like to take a look first.</p>



<a name="242615882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242615882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242615882">(Jun 14 2021 at 15:35)</a>:</h4>
<p>np</p>



<a name="242616048"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242616048" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242616048">(Jun 14 2021 at 15:36)</a>:</h4>
<p>(And actually at first I planned to finish this in May. But it did cost much time to reach consensus before.)</p>



<a name="242616062"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242616062" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242616062">(Jun 14 2021 at 15:36)</a>:</h4>
<p>Yes</p>



<a name="242620882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242620882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242620882">(Jun 14 2021 at 16:06)</a>:</h4>
<p>Although I am willing to, I cannot guarantee that I have time to address the changes before early July. So feel free to push commits if necessary.</p>



<a name="242620900"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242620900" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lrh2000 <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242620900">(Jun 14 2021 at 16:06)</a>:</h4>
<p>Sorry again and hope this won't cause any trouble.</p>



<a name="242634848"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242634848" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mara <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242634848">(Jun 14 2021 at 17:46)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> <a href="https://github.com/rust-lang/rust/issues/85359">#85359</a> is ready for review</p>



<a name="242635644"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/268952-edition%202021/topic/reserved%20prefixes/near/242635644" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/268952-edition-2021/topic/reserved.20prefixes.html#242635644">(Jun 14 2021 at 17:52)</a>:</h4>
<p><span class="user-mention" data-user-id="310399">@Mara</span> <span aria-label="+1" class="emoji emoji-1f44d" role="img" title="+1">:+1:</span></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>